The present invention relates generally to the field of test management, and more particularly to random insertion of pre-requisite instructions for cleanup of unpredictable test results.
System testing checks whether the system under test operates as intended or not. When the system under test does not operate as intended, yielding unexpected results, this is due to defects in the system or the software of the system. The correction of these defects, and the introduction of new features or the enhancement of existing features, may introduce new defects into the system under test. Repeated testing improves the likelihood of catching defects soon after their introduction.
In general, there are two testing approaches with mutually exclusive advantages, namely, manual testing and automated testing. The strength of manual testing, especially in complex software environments, is that the software is used from the point of view of the customer and with a customer-like behavior, thereby executing an arbitrary sequence of activities and inputs. Due to manual testing, the state of the system under test is non-deterministic, e.g., due to user errors. Thus, the system under test might get into a system state that has not yet been considered. Automated testing shows other advantages. Automated testing is less resource intensive than manual testing because no human test person is needed. In addition, occurring failures are deterministically reproducible because of the well-defined run of the test routine. Finally, automated testing is much faster, i.e., a higher coverage of testing is reached in a certain period of time.
Known solutions and tools essentially are focused on automated test execution which provide the advantages of automated testing as described above but do negate the benefits of manual test execution. Hence, once manual test scenarios get automated, most of the mentioned advantages of manual testing are lost because the user-specific randomness is eliminated by using a deterministic test environment. In addition, most automated tests do not focus on long-term effects and do not reflect stochastic, i.e., random, aspects of human user activities.